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TUP PROBLEM 



'•/hat would happen if "freshman" composition were changed to 
"junior" composition? Is there any discernible difference in the 
writing achievement of college juniors who have had composition 
instruction as freshmen, when some have had such instruction as 
juniors, and some have not had it at all? Project 3177 attempts 
to find an answer to this question. 

This Project is an extension of Project 2183: The Ef fecti va- 

ne ss of Col lege - Level Instruct!* on in Freshman Comp os i t i on , which 
involved five uni vers i t i es and was carried on at the University of 
Northern Iowa, and reported in 1969. * That study compared the score 
of students who had completed freshman composition with those of 
students who had not been permitted to register for that course, 
over a two-year period. Two objective tests. Cooperative Engli sh 
Tests : Fngl ish Fx pres si on and Col lege E ntrance Ex ami nat i on Board 

English Compos i + j on Test and a theme, which was a paper written 
wit hi n a two-hour DerfocT on a topic supplied by the investigators, 
were tlv' criterion measures. The students were tested at the begin- 
ning and end of the first semester, at the end of the second 
semester, and at the end of the fourth semester. The scores of the 
two subgrouos, those with freshman composition experience and those 
without it, were compared at each testing period; in addition gain 
scores were compared for the objective tests, and male and female 
scores were compared at each testing period for the total group and 
for each subgroup, including analysis by abi ity quarters, and by 
ability quarters by sex. In Project 2188 a matched pairs design was 
used: at the beginning, students were matched exactly on sex and 

theme score, within one year on age, and within three points on a 
combination of CEEB and COOP scores; that is, three points in a rang 
of roughly 40 to 160 (sum of 7-score for each of the two objective- 
tests) . 

Findings revealed that the scores of 597 students taking 
composition were significantly higher on COOP and theme, but not 
on CEFR, than those for the matched students not taking composition 
at the end of the first semester. Scores of the 365 students taking 
composition were significantly higher on COO^ but not on theme or 
CEE3 at the end of the first year. Scores for the 122 matched pairs 
nor si st i ng at the end of the second year were essential ly the same 
for both subgroups at the end of the sophomore year. Females scored 



■Poss Jewell, John Cowley, and Cordon Rhum, Final Report , 

The E ffect iveness of Col loge - Level Instruct! on i n Freshman Compo - 
sTtion (Cooperative Research Project 2188"), Cedar Falls, Iowa: 
University of Northern Iowa, 1969. 
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better than males on 53 of 54 testing occasions, significantly 
better on 35 of them. It was concluded: "that in investigations 

concerning competence in composition, the ratio between sexes must 
be taken into account in the groups whose performance is being 
studied. Students in the lowest quarter of the control (compo- 
sition) subgroup scored consistently higher on criterion measures 
than those in the lowest quarter of the experimental (non- 
composition) subgroup. 

The present investigators realized, as they reviewed their 
plans for Project 2188, that there would be a number of students 
finishing their sophomore year of college, having been tested four 
times, who had not received instruction of the sort given in 
freshman composition. The opportunity thus presented, of dividing 
the experimentals from Project 2188 into two random groups and, on 
a chance basis, assigning one group to "junior" composition while 
keeping the other group out as experimentals, seemed too good, and 
too unique, to be ignored. 

Project 3177 attempts, then, to determine the effect of "junior" 
composition by comparing the performance of those without freshman 
composition who were enrolled in a junior composition course with 
that of a similar group of students who did not receive composition 
instruction either as freshmen or as juniors. All of the universities 
which cooperated in Project 2188 were invited to participate in 
Project 3177, but only the University of Colorado accepted the invi- 
tati on. ^ 



PROCEDURE 



Because of the much smaller numbers of students (caused partly 
by attrition and partly by the fact that the persisting juniors were 
divided into two groups), analysis of covariance was used in Project 
3177 instead of the matched-pairs design. The analysis by covariance 
procedure provides control on pretreatment differences through 
statistical means rather than experimental. These statistical pro- 
cedures include a "control" variable deriving from a test administered 
at the outset of the investigation and a "criterion" variable deriving 
from a test administered at the conclusion of the investigation. 
Statistically significant changes in the relationship between the 



^Ibid., p. 65. 

^For a summary of related research, see Ibid., pp. 25-37* 
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performances of the groups on the tests at the two testing occasions 
provide evidence concerning the effect of the experimental pro- 
cedures . 

The total of 354 students who had been kept from taking freshman 
composition and who registered for the junior year at both univer- 
sities (the first semester at one university and the second semester 
at the other) were divided into two subgroups on a chance basis. 

The procedure was to list the students’ names, numbering them, and 
then employ a table of random numbers in assigning students either 
to the experimental subgroup (those who would not receive instruction) 
and the control subgroup (those who would receive instruction). As 
in previous selection procedures, males and females were selected 
separately. 

Students assigned to the "junior English" treatment enrolled in 
special composition sections in the fall of their junior year. 
Instructors were told to conduct the class in a manner as similar to 
that in which they conducted their freshman composition classes as 
the maturity of the students permitted. Both these students and the 
"experimental" students were given the three tests — COOP, CEEB, and 
Theme — at the end of the junior semester in which the "control" 
students received instruction in composition. The data from this 
testing was used for the analysis which follows. 

Reliability of Ob j ect i ve Tests* * 

Cooperativ e English Tests ; Engl ish Expre ss i on . This instru- 
ment ,"""pubTi she cf in I960, is composed of two parts: "Part I: 

Effectiveness," thirty items; and "Part II: Mechanics," sixty items. 

The time limits are 15 minutes and 25 minutes respectively. A 
student’s score is the total number of correct responses. This raw 
score is transformed into a Converted Score by means of a table 
provided by the publishers of the test. For Form 1A, the possible 
range in converted scores is from 115 (raw score of 0) to 191 (raw 
score of 90). For the two forms of the test (1A, IB) recommended 
for use with college freshmen and sophomores, the investigators were 
able to find reliability evidence only for the twelfth grade level. 

The correlation between parallel forms was 0.84 and the standard 
error of measurement was on the order of 4.00 converted score units. 

The C ol lege Entrance Exami nati on Board Engl ish Composition 
Test. T hTs Is one of the CEEB achievement tests. Evidence about 



4fhis 



section is quoted from Ibid., 



p. 42. 
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the functioning of this instrument seems to be directly concerned 
with validity. This is reflected in one of the earlier reports on 
the instrument, which appeared with the title "Composition Test 
Shows High Validity on Reliable Criterion of Writing Ability. "5 
The excellent 84-page report called The Measurement of Writing 
Abi 1 ityP also dealt primarily with the validity of the College 
Entrance Examination Board English Composition Test (CEEB). It is 
realized that to achieve validity a test author must at the same 
time achieve reliability. A third source of information was The 
Sixth Mental Measurements Yearbook . Holland Roberts, one of the 
three reviewers of the test, commented on reliability: "For the 

composition test a Kuder-Richardson formula 20 reliability of 0.85 
and a standard error of measurement of 39 is reported, indicating 
satisfactory discrimination among the members of the test group. 



ANALYSIS OF DATA 



Tj, the subgroup which took no work in composition in the 
junior year, was designated the Experimental group. T 2 » the sub- 
group which received composition instruction in its junior year, 
was designated the Control group. As indicated previously, member- 
ship in T •] or T 2 was randomly determined. 

A further division of Tj into males (T 3 ) and females {T 5 ), 
and of T 2 into males (T 4 ) and females (Tg) continued the attention 
to performance by sex which was present in the initial study. 

Project 2188. The reader may find it useful to remember that groups 
with the odd-numbered subscripts (Tj, T^, T 5 ) are experimental; 
those with the even-numbered subscr i pts are controls. 



^"Compositi on Test Shows High Validity on Reliable Criterion 
of Writing Ability , 11 ETS Developments , XI (January, 1963) 1 & 4. 

^Fred Godshalk, Frances Swineford, and William E. Coffman. 

The Measurement of Writing Abi 1 i ty . Mew York: College Entrance 

Examination Boar37 1966. 

^Holland Roberts,Sixth Mental Measurements Yearbook (a review 
of the CEEB English Composition Test), ed. Oscar K. Buros. Highland 
Park, New Jersey: Gryphon Press, 1965» p. 590. 
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The following listings specifically define each symbol used,, 
each treatment used, each hypothesis tested, the control variable,; 
and the criterion variable. 

Symbols Used j 



T : Those students (male and female) labeled as "experimental!" 

' in Project 2188 who entered school in September 1966 as ; 
juniors and were excused from taking instruction in 
composition as juniors ( Experiments 1 ) . 

T • Those students (male and female) labeled as "experimental " 
" in Project 2188 who entered school in September 1966 as I 

juniors and received instruction in composition as 
iuniors (Control). j 

T^: Male ^xperimentals . ! 

i 

T^: Male Controls. 

T^: Female Experimentals . 

Tgt Female Controls. 

Hypotheses Tested 



H : 
1 



There exists no significant difference between T and T^ 
at the conclusion of the treatment period. 



H 



7 ' 



There exists no significant difference between T and T^ 
at the conclusion of the treatment period. 



H 



3 : 



There exists no significant difference between T and 7, 
at the conclusion of the treatment period. ^ ° 



Control Variable 

COOP administered in May 1966. 

Cri teri on Variabl e 

COOP administered at end of treatment period. 



Anal ysi s of Covari ance 

The hypothesis: This hypothesis states that there exists 

no significant difference between T . and T at the end of the treat- 
ment period. This was refuted, using the COOP test as both the 
control and the criterion variable (see Table I). There was a 
significant difference. 
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Results of Test: F=4.7335 (significant at 0.05 level of significance). 



Table II indicates that though there is a significant differ- 
ence between Tj and I 2 * difference arises from the T] 
adjusted Criterion mean decreasi ng whi 1e the adjusted T 2 Criterion 
mean remains essentially the same as the unadjusted criterion, 

TABLE II 



COMPARISON OF THE CONTROL, CRITERION, AND ADJUSTED 
CRITERION MEANS ON 1967 COOP SCORES 



Means 


T | (N = 86) 


Control (May, 1966 ) 


169.70 


Criterion (Jan., 1967) 


166.02 


Adjusted Criterion 


165.99 



T ? (N = 143) 

169.55 

168.08 

168.10 



When the analysis of covariance was repeated, using male 
students only, the hypothesis: there exists no significant differ- 

ence between T 3 and T/^ at the conclusion of the treatment period, 
using COOP as Both the control and the criterion variable, was also 
refuted (see Table III). 



Table IV indicates that the significant difference between T^ 
and T/j is duo largely to the greater decrease between the Control 
and Criterion iiieans of T^. 

A test for the homogeneity of regression indicates that the 
F-ratio used for testing the H 2 hypothesis did not satisfy the F- 
distribution for the stated degrees of freedom. ^ There exists some 
evidence, however, that the F-ratio tends to be conservative when 
the assumptions underlying the covariance model are not met and thus 
would probably not affect this result (F = 11. 7 + ).^ 

When the analysis of covariance design was applied to the female 
group to test the hypothesis: there exists no significant differ- 

ence between Tc and Tg at the conclusion of the testing period, using 
COOP as both tne control and the criterion variable, the null hypo- 
thesis was upheld (Table V). 



®E . F. Lindquist, Design and Analysis of Experiments in 
Psychology and Education (Boston, Mass.: Houghton Mifflin, 1956), 

pp. 330-3 T. 

O 

Percy D. Peckham, 'The Robustness of the Analysis of Covariance 
to Various Regression Slopes," University of Washington. Paper pre- 
sented at the Annual AERA Conference, 1970. 



9 

11 









X 


CM 


CA 










03 




o 










•M 


9 


• 










t/J 


o 


00 










3 


00 


LA 










••n 


VO 








LU 




>- X 










O 




00 < 










2 














< 














HH 














8E 




M- 

O 










> 














O 




co E 










o 




<u o 

03 X 


r— 


ON 


O 




Ll 




k- (V 




00 


A 




o 




CD 0) 

03 L- 










GO 




Q Ll 










HH 














00 














>- 














_l 




*>. 










< 




X 


CM 


r-* 


A 




2 




03 




CM 






< 




«M 


• 


• 


• 




o 




VJ 

3 


o 

CO 




-d- 

-d- 




2 




l 


VO 


•— 


00 




HH 




>- XI 




•v 


9k 




00 




00 C 




A 


A 








00 ^ 










to 














LU 














QT 


c 












O 


o 


r^ 


CO 


00 


VO 




o 


*F“ 


VO 


CO 


ON 


00 




00 


• L. 


\ 


• 


• 


• 






<u 


r— 


LA 


ON 


A 




O- 


4-» 




LA 


<A 


A 




o 


•r- 


a. 


CO 


00 


VO 




o 




o 




* 


* 


»— 1 


o 


o 


00 o 




A 


VO 


M 




'■w' 


to o 








HH 


1^ 














VO 












LU 


ON 












_I 


•— 






00 


r— 


A 


CO 






tn 


LA 


A 


O 


< 


2 




•M 


• 


• 


• 


H 


O 




O 


VO 


CA 


o 








3 


A 




(A 




-4- 

H 




*8 


CN 


A 

•s 


CM 

•t 








L_ 




r— 


CM 




Q 




O. 










2 




00 ** — ' 










< 














C"> 






CM 


A 


r-* 




H 


<"— N 


VO 


ON 


A 


00 








VO 


• 


• 


• 




£ 


o 


*\ 


VO 


m 


o 




u 


LA 


r-* 


VO 


-d- 






•M 






r^ 


00 




o 


C 


a. 




9k 


9k 




cc 


o 


o 




A 


A 




o 


O 


00 o 














00 o 










UJ 














u 




M- 














O 










Ll 




V) E 










o 




cu o 

a) X 


?— 


O 


, 




2 




L- Q) 




A 


A 




O 




cn a 










00 




03 L. 










HH 




O Ll 










or 














< 














Q- 






in 
















VI 






O 






c 


4-» 






o 






03 


C 












B 


a> 












4-» 


£ 












03 


•M 












03 


03 












L. 


03 












H 


1. 














H 










CO 


c 












03 


03 


c 










u 


03 


• r- 










L. 


5 


-C 


03 








3 


4J 


U» 


•M 








O 


03 


•#- 


o 








CO 


oc 




1- 




ia 

12 



Results of Test: F=1 1 . 72 1 1 (significant at 0.05 level of significance) 



TABLE IV 



COMPARISON OF THE CONTROL, CRITERION, AND ADJUSTED CRITERION 
MEANS FOR TWO MALE GROUPS ON 1967 COOP SCORES 



Means 


t 3 (n = 30 ) 


T 4 (N = 62) 


Control (May, 1966) 


167.03 


168.98 


Criterion (Jan., 1967) 


160.30 


166.81 


Adjusted Criterion 


160.75 


166.59 



The three hypotheses, Hj, H 2 , and H^, were again tested using 
the same statistical design, but with the CEEB test as the Control 
and Criterion variables. The null hypothesis, in all three analyses, 
was upheld (see Tables VI, VII, and VIII). 

The test of homogeneity of regression did reveal a significant 
difference for the male group. It is doubtful, however, that a real 
difference exists between T 3 and Tj, considering that the F-ratio was 
quite small (F = 0.80) and the conservative tendency of the F-ratio 
when the assumptions of the design are not met. 

The same analysis was used to test the three hypotheses again, 
this time using the Theme scores as the Control and Criterion vari- 
ables. In all cases, the test upheld the hypotheses (see Tables IX, 
X, and XI). 

Tables XII and XIII summarize the results obtained from testing 
the Hi, H 2 , and H 3 hypotheses for each analysis, using the three 
Criterion and Control variables, i.e., COOP, CEE 8 , and THEME. 

It is noted in Table XIII for the CEEB column that a difference 
of 14.42 score points exists between To and T^ for the male group. 

A similar difference is noted between the female groups (T^, Tg), but 
in the opposite direction. Yet, no statistically significant differ- 
ence was observed. Table XIV shows the large score variability for 
the CEEB variable, disclosing the apparent reason for no significant 
di fference. 

Table XIV summarizes the criterion score variability for each 
variable. 
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Results of Test: F-0.0082 



TABLE XII 



SUMMARY OF RESULTS USING THE ANALYSIS OF COVARIANCE APPLIED TO 

THE THREE HYPOTHESES 



Group 


COOP 


d.f . 


CEEB 


d.f. 


THEME 


d.f. 


Total 


4.7335* 


1 & 226 


0.1671 


1 & 230 


0.1722 


1 & 234 


Male 


11.7211* 


CT\ 

00 

ub 


0.7954 


1 & 91 


3.0206 


1 &■: 95 


Female 


0.0667 


1 & 134 


0.0822 


1 & 136 


0.0082 


1 &' 136 



■’’•Significant at 0.05 level of significance. 



TABLE XIII 

COMPARISON OF THE ADJUSTED CRITERION MEANS FOR EACH OF THE VARIABLES 
COOP CEEB THEME 



Group 


*1 


'2 


3 


'4 


'5 


'6 


Total 


165.99 


168.10 


535.97 


532.05 


9.13 


9.26 


Male 


160.75 


166.59 


500.88 


514.46 


7.80 


8.72 


Female 


168.91 


169.19 


556.95 


544.14 


9.79 


9.76 



TABLE XIV 

COMPARISON OF THE CRITERION SCORE VARIABILITY FOR 
THE EXPERIMENTAL AND CONTROL GROUPS 

COOP CEEB THEME 



Group 


Exp. 


Cont . 


Exp. 


Cont. 


Exp. 


Cont. 


Total 


7.46 


7.61 


89.43 


96.19 


2.64 


2.59 


Male 


6.74 


8.50 


72.57 


104.54 


2.44 


2.50 


Female 


5.85 


6.69 


88.35 


87.74 


2.37 


2.50 
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SUMMARY 



The research attempted was to determine whether or not there 
was any difference between the scores of two groups of juniors, 
both of which had had no freshman composition instruction, but one 
of which had had composition instruction in the junior year and the 
other had not. Three measures, COOP, CEEB, and a theme, were 
administered at the end of the sophomore year and again at the time 
when those juniors who took a composition course as juniors completed 
the course. Comparisons were made between scores of the two groups 
(composition and non-composition); of the males of each group, and 
of the females of each group. 

The results of the research were inconclusive. Of the nine 
test analyses (the two total groups, the males in each group, the 
females in each group on each of three tests), there was a signi- 
ficant difference in only two, both on the COOP test; (1) the 
juniors taking composition scored significantly better on the final 
testing on COOP than did those juniors not taking composition; (2) 
the junior males who had composition scored higher than the junior 
males who did not have composition. On the other seven test analyses, 
there was not a significant difference. 

The following statements summarize the findings: 

1. There was a statistically significant difference in favor 
of those juniors taking junior composition as measured by 
the COOP test. This resulted not from those having 
composition achieving a higher average score than before 
composition, but from their remaining about the same and 
the non-composition group decreasing in average score. The 
investigators cannot account for this. 

2. There was a statistically significant difference in favor 
of the males taking junior composition over the males not 
taking composition as measured by the COOP test. This 
difference resulted also from the fact that the scores of 
those not taking junior composition decreased more than 
those of the group taking junior composition. 

3. The significant difference referred to in 1. above is due 
to the male group. The difference in scores of the two 
female groups was so slight that most of the difference 
between the composition and the non-composition groups is 
accounted for by the males. 
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4. There was no significant difference between the compo- 
sition and the non-composition groups, either as a whole, 
or as males, or as females as measured by both CEE3 and 
a theme. The differences between the composition and 
non-composition groups on CEEB was large (12 for females 
and 14 for males), but that difference was not significant. 
The investigators cannot explain this. 

The results of this research are inconclusive so far as the 
value of a composition course at the junior year is concerned. There 
seems to be no explanation for the drop in scores. Since the investi- 
gators are not in a position to assert that any one test of writing 
skill is better or more reliable than any other, the significant 
difference disclosed by one test cannot be used as the sole or the 
main determinant. On the other hand, those advocating that the junior 
year is a better place for composition than the freshman year cannot 
take much comfort from the results recorded here. 
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APPENDIX 



(Theme Instructions for May 1966) 



THEME INSTRUCTIONS 

1. The paper which you are about to write will be judged on your 
success in presenting your thoughts in a clear, unified, well- 
organized manner, observing the conventions of standard written 
English. You should think about the topic until you have deter- 
mined what idea you want to convey to the reader and the general 
procedure you will follow in doing so. Then you may write your 
paper. Do not hesitate to make a brief outline tHF_ you desi re to 
do s_o (use the back of this sheet). An outline is not required. 

?. You should write as neatly and legibly as you can, but you should 
not hesitate to make changes between the lines if you believe 
them to be necessary. You do not have, to copy the paper over. 

3. WRITE ON ONE SIDE OF THE PAPER ONLY. If you need more paper, 
ask for it. 

4. Begin on the third line of the first sheet, and WRITE ON EVERY 
LIEF THEREAFTER. 

5. You must write with INK or BALL-POINT PEN. 

6. Be certain to write your STUDENT NUMBER in the blank provided 
at the top of this instruction sheet in the upper left-hand 
corner under the Total Score box. It should also be written 
on each page of your theme. Oo^ NOT write your name , or the 
name of your school, in any place other t han the blank provi ded 
at the~bottom~~of thi s sheet . 

7. Turn in all of the paper given to you. 

8. You must stay at least one hour and fifteen minutes. 

9. LENGTH: 300 - 500 words. 

TOPIC 

Convent i ona 1 is a word frequently used to refer to customary 
attitudes, beliefs or actions. In the United States it is a 
convention for men to be clean-shaven, women to wear a certain 
amount of make-up, boys to be interested in sports, and girls to 
be interested in becoming wives and mothers. A person who is 
unconvent i ona 1 in some way departs from the conventions of action 
or bel i eT of the society of which he is a part. 
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With this explanation in mind, discuss the following statement: 

"Convention is society's safeguard, but also its potential 
ex ecut i oner, " To what extent and in what ways do you agree with 
this statement? Use examples and details from your knowledge and 
experience to support your conclusion. 



(Theme Instructions for January, 1967) 
THEME INSTRUCTIONS 



1. The paper which you are about to write will be judged on your 
success in presenting your thoughts in a clear, unified, well- 
organized manner, observing the conventions of standard written 
English. You should think about the topic until you have 
determined what idea you want to convey to the reader and the 
general procedure you will follow in doing so. Then you may 
write your paper. Do not hesitate to make a brief outline i f 
you des ire to do sjd (use the back of this sheet). An outline 
is not requ ire <JT 

2. You should write as neatly and legibly as you can, but you 
should not hesitate to make changes between the lines if you 
believe them to be necessary. You do not have to copy the 
paper over. 

3. WRITE ON ONE SIDE OF THE PAPER ONLY. If you need more paper, 
ask for it. 

4. Begin on the third line of the first sheet, and WRITE ON EVERY 
LINE THEREAFTER. 

5. You must write with INK or BALL-POINT PEN. 

6. Be certain to write your STUDENT NUMBER in the blank provided 
at the top of this instruction sheet in the upper left-hand 
corner under the Total Score box. It should also be written 
on each page of your theme. To NOT write your name , or the 
name of your school, in any place other than the blank provi ded 
at the”"bottom of tlhis~sheet. 



7. Turn in all of the paper given to you. 

8. You must stay at least one hour and fifteen minutes. 

9.. LENGTH: 300 - 500 words. 



TOPIC 

Thoreau, watching his neighbor plowing observed that the farm 
owned the farmer, rather than the reverse, as the farm demanded all 
of the farmer's time and energy. Noting other instances of a similar 
nature, Thoreau concluded: 

. 'Things are in the saddle and ride mankind." 

Today, in order to "own" automobiles, lawnmowers, homes, boats, 
and other things, many men are forced to "moonlight" and many wives 
to work. 
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Consider the contemporary situation from the point 
of Thoreau's statement. Form an opinion concerning the 
ship of mankind to things. Then write a paper in which 
your opinion, supporting it from your own knowledge, ex 
and observation. 



of view 
relati on- 
you give 
per ie nee. 
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